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BACKGROUND OF THE INVENTION 

1. FIELD OF THE INVENTION: 

The present invention relates to a vision sensor 
5 capable of omnidirectional observation encompassing a 
viewing range 360 degrees around the vision sensor. In 
particular, the present invention relates to an 
omnidirectional vision sensor which is used in a vision 
system for a monitoring camera system or a mobile robot, 
10 etc*, and which can obtain field of view information 
associated with the entire surroundings in real-time, 

2. DESCRIPTION OF THE RELATED ART: 

In recent years, various omnidirectional vision 
15 sensors have been proposed as input devices through which 
visual information covering a broad range is to be input 
to any device coupled to the omnidirectional vision 
sensors, with a view to developing applications for 
monitoring camera systems or mobile robots, etc. 

20 

The following techniques are known, for example: 
CD a method in which images which are captured by means 
of a single rotating camera are linked together 
(Japanese Laid-Open Publication No. 10-105840, 
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entitled "System for automatically detecting an 
intruding object" ) ; 

CD a method in which images which are captured by means 
of a rotating plate mirror are linked together (Japanese 
Laid-Open Publication No. 11-4373, entitled "Method 
and apparatus for constructing omnidirectional 
panoramic images"); 

(D a method in which omnidirectional images are captured 
at one time by means of a plurality of fixed cameras 
(Japanese Laid-Open Publication No. 11-164292, 
entitled "Image generation device, image presentation 
device, image generation method and image synthesis 
method" ) ; 

<D a method in which an image from a wide field of view 
is captured at one time by means of a wide-angle lens 
such as a fish-eye lens (Japanese Laid-Open Publication 
No. 11-355763, entitled "Monitor system and monitor 
method" ) ; and 

<D a method in which an image is captured at one time 
by means of a reflection mirror of a special shape such 
as a spherical, conical, hyperbolic, etc., shape 
(Japanese Laid-Open Publication No. 11-218409, 
entitled "Method and apparatus for measuring three- 
dimensional information" ) . 
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Method (D mentioned above involves acquiring 
images of the surroundings by means of a single television 
camera which is placed on an electrically actuated base 
and is rotated by 360 degrees, where the images are linked 
together by image processing- By using this method, it 
is possible to acquire omnidirectional images with a 
relatively high resolution. However, since the camera 
is rotated while acquiring images, it is impossible to 
acquire omnidirectional images at one time, thus the 
resultant image is no longer a real-time image. 

Method ® mentioned above involves rotating a 
mirror by 360 degrees so as to acquire images of the 
surroundings which are reflected by the mirror are 
captured by means of a fixed camera, where the images are 
linked together by image processing . Thus , it is possible 
to acquire omnidirectional images with a relatively high 
resolution, as is the case with method®. However, since 
the mirror is rotated while acquiring images, it is 
impossible to acquire omnidirectional images at one time, 
thus resultant image is no longer a real-time image, as 
is the case with method CD. 
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Methods CD and ® mentioned above utilize a 
mechanical means for rotating a camera or a mirror, 
respectively, thus requiring some sort of maintenance 
work for the mechanical means in order to enable operation 
over a long period of time. Accordingly, methods ® to 
(5) mentioned above have been proposed as methods which 
enable a one-time acquisition of omnidirectional images 
without employing any mechanical means . 

Method ® mentioned above involves acquiring 
omnidirectional images at one time by employing a 
plurality of fixed cameras, and is advantageous from the 
perspective of obtaining images in real-time. Moreover, 
since no special mechanical means is required, this method 
is suitable for long periods of use, and provides for good 
reliability. However, there is a problem in that the use 
of a plurality of camera leads to an increased system cost . 

Methods @ and © mentioned above employ a 
wide-angle lens or a reflection mirror of a specific shape 
to enable a one-time acquisition of an image from a wide 
field of view. As is the case with method (3) mentioned 
above, this method is advantageous from the perspective 
of obtaining images in real-time, and, since no special 
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mechanical means is required, this method is suitable for 
long periods of use and provides for good reliability. 
Furthermore, unlike method (3) mentioned above, only one 
camera is required, thereby reducing the system cost. 
However, with methods ® and © f it is impossible to 
acquire complete omnidirectional images encompassing 
360 degrees. In other words, the resultant field of view 
includes a blind spot(s). 

Hereinafter, the field of view and blind spots 
which are inherent in methods ® and <D mentioned above 
will be described with reference to Figures 6 to 10 . Each 
plane of Figures 6 to 10 is a vertical plane which contains 
a central axis therein, with a camera being disposed below 
a lens or a mirror. 

Figure 6 illustrates a f ield of view in the case 
where a wide-angle lens 10 is employed in method (D 
mentioned above. When the system is constructed so that 
the wide-angle lens 10 is disposed with its convex portion 
"up" (as shown in Figure 6), with the imaging means 
including a camera being located below the wide-angle 
lens 10, it would be possible to acquire an image from 
the space above a horizontal plane extending 360 degrees 
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around the lens, an image of only an upper half of the 
surrounding sphere along the vertical direction can be 
captured. That is, the lower half of the surrounding 
sphere is left as a blind spot. 

5 

Figure 7 illustrates a field of view in the case 
where a conical mirror 20 is employed as a body- of - 
revolution mirror in method® mentioned above. While the 
images captured by this method encompass a horizontal span 
10 covering 360 degrees around the mirror, the mirror face 
presents an obstacle along the vertical direction, 
creating a blind spot above and below the horizontal span. 
In other words, a blind spot exists in the "front" of the 
camera (imaging means). 

15 

Figure 8 illustrates a field of view in the case 
where a spherical mirror 30 is employed as a body-of- 
revolution mirror in method (D mentioned above. While the 
images captured by this method encompass a horizontal span 
20 covering 360 degrees around the mirror, the mirror face 
presents an obstacle along the vertical direction, 
creating a blind spot above the horizontal span. In other 
words, a blind spot exists in the "front" of the camera 
(imaging means). 
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Figure 9 illustrates a field of view in the case 
where a hyperbolic mirror 40 is employed as a body- 
of -revolution mirror in method® mentioned above. While 
5 the images captured by this method encompass a horizontal 
span covering 360 degrees around the mirror, the mirror 
face presents an obstacle along the vertical direction, 
creating a blind spot above the horizontal span. In other 
words, a blind spot exists in the "front" of the camera 
10 (imaging means). 

Figure 10 illustrates a field of view in the case 
where a parabolic mirror 50 is employed as a body-of- 
revolution mirror in method® mentioned above . While the 

15 images captured by this method encompass a horizontal span 
covering 360 degrees around the mirror, the mirror face 
presents an obstacle along the vertical direction, 
creating a blind spot above the horizontal span. In other 
words, a blind spot exists in the "front" of the camera 

20 (imaging means). 

Thus , according to any of methods @ and (D 
mentioned above, blind spots exist in a portion of the 
field of view. Method® is also disadvantageous in that 
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the resultant field of view only expands in the upper 
direction, as described above. Therefore, when method 
® is implemented in a mobile robot, for example, only the 
ceiling of a building which accommodates the mobile robot 
would always be observed. Thus, method® does not enable 
sufficient observation of the lateral directions of the 
robot, which is essential in preventing the robot from 
colliding with other objects, while only permitting the 
observation of the upper region which does not require 
much attention. 

In recent years, method (B mentioned above, which 
involves the use of a body-of -revolution mirror, has 
attracted much attention because this method is 
advantageous from the perspective of obtaining images in 
real-time, low cost, and high reliability, in spite of 
some blind spots in a portion of the field of view. In 
particular, when method (E> is implemented with a 
hyperbolic mirror (among other bodies-of -revolution ) , an 
optical system of a perspective projection type is 
typically used. As a result, the obtained image can be 
easily converted to an image as seen from a focal point 
of the mirror (which should appear similar to an image 
which is imaged via a common camera), or an image which 
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would be obtained by rotating a camera along a vertical 
axis (a cylindrical omnidirectional image). Thus, a 
greater variety of image processing is possible than any 
other method which employs mirrors . An omnidirectional 
visual system employing a hyperbolic mirror is described 
in Japanese Laid-Open Publication No. 6-295333. 

However, in any variant of method (D mentioned 
above, blind spots exist in the frontal direction of the 
camera (imaging means). In order to be able to apply 
omnidirectional vision sensors to a wide range of usage, 
it would be desirable to further reduce the blind spots. 

SUMMARY OF THE INVENTION 

According to the present invention, there is 
provided an omnidirectional vision sensor comprising: an 
optical system including a body-of -revolution mirror 
having a convex portion and having a symmetrical structure 
with respect to a revolution axis, wherein the body- 
of -revolution mirror includes a cutaway section in the 
convex portion of the body-of -revolution mirror so as to 
allow light incident from surroundings of the revolution 
axis of the body-of -revolution mirror to be collected; 
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and imaging means, including a light -receiving element 
for receiving the collected light and image processing 
means for converting an optical image generated from the 
collected light received by the light -receiving element 
into image data, wherein the revolution axis of the 
body-of -revolution mirror and an optic axis of the 
light -receiving element coincide. 

According to the above structure of the present 
invention, an area in the frontal direction of an imaging 
means (i.e., an upper direction of the optical system), 
which would be a blind spot in conventional structures, 
also becomes part of the field of view. Thus, according 
to the present invention, the field of view is expanded, 
as will be described more specifically in an embodiment 
to follow. 

In one embodiment of the present invention, the 
optical system further comprises a wide-angle lens 
provided in the cutaway section of the body-of -revolution 
mirror, the wide-angle lens being disposed so that a 
convex portion of the wide-angle lens faces away from the 
imaging means . 
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According to the above structure of the present 
invention, the field of view can be expanded base on a 
further reduction of the blind spot, as will be described 
more specifically in an embodiment to follow. 

5 

In another embodiment of the present invention, 
a field of view of the wide-angle lens coincides with a 
blind spot of the body-of -revolution mirror. 

10 According to the above structure of the present 

invention, the blind spot in the upper direction can be 
eliminated, as will be described more specifically in an 
embodiment to follow. 



15 Thus, the invention described herein makes 

possible the advantages of providing an omnidirectional 
vision sensor which can be used in a wide range of 
applications, with substantially reduced blind spots 
compared with those associated with conventional 

20 omnidirectional vision sensors employing a body-of - 
revolution mirror (e.g., a conical mirror, a spherical 
mirror, a hyperbolic mirror, or a parabolic mirror) . 



This and other advantages of the present 
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invention will become apparent to those skilled in the 
art upon reading and understanding the following detailed 
description with reference to the accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a diagram illustrating an expanded 
field of view in accordance with an omnidirectional vision 
sensor according to one e embodiment of the present 
invention. 

Figure 2 is a diagram illustrating an expanded 
field of view in accordance with an omnidirectional vision 
sensor according to another embodiment of the present 
invention . 

Figure 3 is a diagram illustrating images 
obtained on an imaging plane of a camera in accordance 
with an omnidirectional vision sensor according to one 
embodiment of the present invention. 

Figure 4 is a diagram illustrating relative 
positions of a hyperbolic mirror and a camera in a 
conventional structure . 
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Figure 5 is a diagram illustrating imaging 
relationship between a camera lens and a wide-angle lens. 

5 Figure 6 is a diagram illustrating a field of view 

and blind spots in a conventional structure employing a 
wide-angle lens. 

Figure 7 is a diagram illustrating a field of view 
10 and blind spots in a conventional structure employing a 
conical mirror. 

Figure 8 is a diagram illustrating a field of view 
and blind spots in a conventional structure employing a 
15 spherical mirror. 

Figure 9 is a diagram illustrating a field of view 
and blind spots in a conventional structure employing a 
hyperbolic mirror. 

20 

Figure 10 is a diagram illustrating a field of 
view and blind spots in a conventional structure employing 
a parabolic mirror. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Now, the present invention will be described by 
way of illustrative examples, with reference to the 
5 accompanying figures* 

Hereinafter, the word "upper" or "above" will be 
employed to mean a direction in which a body-of -revolution 
mirror is disposed relatives to a camera, while "lower" 

10 or "below" means the opposite direction thereof. Thus, 
a body-of -revolution mirror will be referred to as being 
disposed in an "upper" direction of or "above" a camera, 
whereas the camera will be referred to as being disposed 
in a "lower" direction of or "below" the body-of- 

15 revolution mirror. 

( Embodiment 1 ) 

Figure 1 is a diagram illustrating an 
omnidirectional vision sensor according to an embodiment 
20 of the present invention and a field of view provided 
thereby. The omnidirectional vision sensor includes a 
body-of -revolution mirror 60 and an imaging means 64. 
The body-of -revolution mirror 60 has a cutaway section 62 
in the center of a convex portion thereof. The imaging 
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means 64 includes: a camera 61 which has a light- 
receiving element (e.g., a camera lens) and an image 
processing means 63. According to the present invention, 
a body-of -revolution mirror is used as an exemplary 
5 optical system. 

According to the present embodiment of the 
invention, an optical image which is generated by light 
reflected from the body-of -revolution mirror 60 and an 

10 optical image which is generated by light passing through 
the cutaway section 62 of the body-of -revolution 
mirror 60 (collectively referred as light which is 
incident from the surroundings of the revolution axis) 
can be converted to image data, which is processed by the 

15 image processing means 63 in the imaging means 64 to give 
real-time images . 

In each of the conventional omnidirectional 
vision sensors illustrated in Figures 7 to 10, where a 
20 body-of -revolution mirror (e.g., a conical mirror, 
spherical mirror, a hyperbolic mirror, a parabolic 
mirror) is employed as an optical system which provides 
a field of view encompassing a horizontal span covering 
360 degrees around the lens, the mirror face presents an 
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obstacle creating a blind spot in the frontal direction 
of the imaging means (a camera, etc. ) . On the other hand, 
the area below the mirror, which constitutes a part of 
the field of view, will only accommodate the camera and 
other elements of the imaging means. 

In contrast, according to the present invention, 
as shown in Figure 1, for example, the frontal direction 
of the camera 61 (the upper direction of the optical 
system) can be seen through the cutaway section 62 
provided in a central portion of the convex portion of 
the body-of -revolution mirror 60, which is disposed so 
that the convex portion faces the camera 61* Thus, 
according to the present invention, the field of view can 
be expanded above the optical system where there is 
typically a blind spot in conventional structures. 
Although a blind spot is present in the rear direction 
of the camera 61 (i.e. , the lower direction of the optical 
system) , this only means that the camera 61 and the other 
elements in the imaging means are not captured within the 
field of view, which is not particularly problematic. The 
size of the cutaway section 62 can take various preferable 
ranges depending on the required angle of view. However, 
the cutaway section 62 preferably has a size at least 
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equivalent to the field of view of the camera itself. 

Furthermore, a wide-angle lens may be placed at 
the cutaway section 62 of the body-of -revolution 
mirror 60, with its convex portion facing away from the 
camera 61 # to further reduce the blind spot in the frontal 
direction of the camera and expanding the field of view. 
This embodiment is shown in Figure 2, as discussed below. 

( Embodiment 2 ) 

Figure 2 is a diagram illustrating an 
omnidirectional vision sensor according to another 
embodiment of the present invention and a field of view 
provided thereby. The omnidirectional vision 

sensor includes a body-of -revolution mirror 70 and an 
imaging means 74. The body-of -revolution mirror 70 has 
a cutaway section 62 in the center of a convex portion 
thereof, with a wide-angle lens 72 being provided in the 
cutaway section 62. The imaging means 74 includes: a 
camera 71 which has a light-receiving element (e.g., a 
camera lens) and an image processing means 73. 

As shown in Figure 2, by selecting and positioning 
the wide-angle lens 72 so that the range of the field of 
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view (angle of view) of the wide-angle lens 72 coincides 
with the blind spot associated with the body-of - 
revolution mirror 70, the blind spot in the frontal 
direction of the camera 71 can be substantially 
5 eliminated* 

Figure 3 shows images obtained on an imaging plane 
of a camera in accordance with an omnidirectional vision 
sensor according to the present invention. Since a 

10 body- of -revolution mirror is employed in the optical 
system according to the present embodiment of the 
invention, two concentric circular images are obtained. 
Specifically, an image which appears in an outer region 80 
is an image of a horizontal span covering 360 degrees 

15 around the body-of -revolution mirror, obtained via the 
body-of -revolution mirror. An image which appears in an 
inner region 81 is an image of an area lying in the frontal 
direction of the camera which is obtained through the 
cutaway section of the body-of -revolution mirror. Thus, 

20 according to the present invention, an image of a 
horizontal span covering 360 degrees around and an image 
lying in the frontal direction of the camera can be 
simultaneously obtained. These two images may be linked 
together by being subjected to a panoramic conversion 
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using conventionally-known image processing techniques, 
so that image information concerning an area which is 
conventionally a blind spot can be concurrently captured. 
Thus, images of a broader portion of the surroundings can 
be obtained in real-time according to the present 
embodiment of the invention. 

In the above description, any reference to an 
"image" being captured by means of a "camera" is 
equivalent to a "still image" being captured by means of 
a camera . It will be appreciated that a "video image (i.e., 
moving images ) " can be captured by means of a video camera. 
For example, by acquiring 30 still images per second and 
subjecting these still images to appropriate image 
processing, it would be possible to obtain a "video image 
(i.e., moving images)". As described above, the present 
invention makes it possible to acquire an omnidirectional 
image at one time in a real-time manner. In the following 
description, it should be appreciated that any reference 
to a "camera" and an "image" may also be read as a "video 
camera" and a "video image", respectively. 

(Embodiments employing a hyperbolic mirror) 

Hereinafter, specific embodiments of the present 
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invention, in which a hyperbolic mirror is employed as 
a body-of -revolution mirror, will be described in detail. 

A conventional omnidirectional visual system 
5 employing a hyperbolic mirror is specifically described 
in Japanese Laid-Open Publication No. 6-295333 f as 
mentioned above. As shown in Figure 4, a hyperbolic 
mirror 90 and a camera are positioned so that a revolution 
axis (central axis) of the hyperbolic mirror 90 coincides 

10 with an optic axis of a light -receiving element (i.e., 
a lens 91) of the camera. The center of the camera 
lens 91 is placed at a second focal point 93 of the 
hyperbolic mirror 90. An imaging plane 94 is set so as 
to be away from the camera lens 91 by a distance equal 

15 to a focal length of the camera lens 91. Thus, image 
information representing a horizontal span covering 
360 degrees around a first focal point 92 of the 
hyperbolic mirror 90 is imaged on the imaging plane 94. 

20 With reference to Figure 4, consider a coordinate 

system having an X axis and a Y axis extending in a 
horizontal plane (i.e., a plane parallel to the imaging 
plane 94), a Z axis extending in a vertical direction, 
and an origin which is defined as an intersection between 
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asymptotes 95 and 96 of the hyperbolic mirror 90. 
According to this coordinate system, the surface of the 
hyperbolic mirror 90 can be represented by eq. 1: 



5 (X2+Y2)/a2-Z2/b2=-l ... eq. 1. 

In eq. 1, a and b are parameters for determining 
the shape of the hyperbolic. A distance c from the 
intersection between the asymptotes 95 and 96 to the first 
10 focal point 92 and the second focal point 93 of the 
hyperbolic mirror 90 can be represented by eq. 2: 

c=J~ (a2+b2) ... eq. 2. 



15 According to the present invention, a hyperbolic 

mirror 60 having a cutaway section 62 in the center of 
a convex portion thereof as shown in Figure 1 is employed 
instead of the regularly- shaped hyperbolic mirror 90 
shown in Figure 4, along with the camera 61 and the image 

20 processing means 64 shown in Figure 1. As a result, as 
shown in Figure 3, an image which is similar to that 
directly taken by means of a common camera appears in the 
inner region 81. 
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In the above construction, there may still be a 
blind spot in a portion of the area lying in the frontal 
direction of the camera as shown in Figure 1. In order 
to further reduce this blind spot, a wide-angle lens 72 
can be provided in the cutaway section as shown in Figure 2, 
thereby obtaining a further expanded field of view. 
Hereinafter, an exemplary design of the wide-angle 
lens 72 will be described. 

Figure 5 is a diagram illustrating relative 
positions of a camera lens 100 and a wide-angle lens 101. 
The wide-angle lens 101 shown in Figure 5 is a wide-angle 
lens which is to be placed in a cutaway section of a 
hyperbolic mirror according to the present invention. An 
image of an object point 102 is converged by the 
wide-angle lens 101 at a first imaging point 103. The 
image at the first imaging point 103 is converged by the 
camera lens 100 at an image point 104. Assuming that the 
camera lens 100 has a focal length fl; the wide-angle 
lens 101 has a focal length f2; the distance between the 
camera lens 100 and the image point 104 is SI; the 
distance between the camera lens 100 and the first imaging 
point 103 is S2; the distance between the wide-angle 
lens 101 and the first imaging point 103 is S3; and the 
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distance between the wide-angle lens 101 and the object 
point 102 is S4 / imaging formulae of the respective 
lenses 100 and 101 can be represented as: 

5 l/Sl+l/S2=l/f 1 ... eq. 3; and 

l/S3+l/S4=l/f 2 ... eq. 4. 

Now, assuming: 

10 S2+S3=d ... eq. 5 , 

by constructing an omnidirectional vision sensor 
incorporating a hyperbolic mirror which has a cutaway 
section accommodating the wide-angle lens 101, such that 

15 the center of the wide-angle lens 101 is placed at a 
distance d from the camera lens 91 disposed at the second 
focal point 93 of the hyperbolic mirror 90 in Figure 4, 
an image which is obtained through the wide-angle lens 
will appear as an image in the inner region 81 as 

20 illustrated in Figure 3. 

Furthermore, in Figure 5, the optical system 
consisting of the camera lens 100 and the wide-angle 
lens 101 may be regarded as one synthetic lens system, 
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having a focal length f which is defined as: 

l/f=(f l + f2-d)/f lXf 2 ... eq. 6. 

5 Assuming that the synthetic lens system has a 

diameter D, its angle of view n can be defined as: 

n=D/f ... eq. 7. 

10 Now, in order to ensure that the angle of view of 

the wide-angle lens coincides with the blind spot of the 
hyperbolic mirror, the following relationship must be 
satisfied: 

15 n=D/f=b/2Xa ... eq. 8. 

In eq. 8, a and b are fixed for a given shape of 
the hyperbolic mirror; fl is fixed for the given camera 
lens 100; and d is determined in accordance with the 
20 diameter of the wide-angle lens 101. Therefore, by 
calculating f2 from eq. 6 to eq. 8, a wide-angle lens to 
be used for an omnidirectional vision sensor which is 
substantially free of blind spots can be designed. 
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Although the above embodiments illustrate the 
case where a hyperbolic mirror is employed as a body- 
of -revolution mirror, an omnidirectional vision sensor 
according to the present invention can be realized by 
5 employing any other type of body-of -revolution mirror, 
e.g* , a conical mirror, a spherical mirror, or a parabolic 
mirror . 

For example, an omnidirectional vision sensor 
10 incorporating a body-of -revolution mirror which has a 
cutaway section can be realize by selecting the depth of 
field of a camera so that the field of view information 
obtained via the cutaway section is converged on the 
imaging plane of the camera. A wide-angle lens may be 
15 further incorporated in the omnidirectional vision sensor, 
the wide-angle lens being disposed at a distance as 
calculated in accordance with eq. 5. Furthermore, an 
omnidirectional vision sensor in which an angle of view 
of the wide-angle lens coincides with a blind spot of the 
20 body-of -revolution mirror can be realized by, for a given 
blind spot of the body-of -revolution mirror used, 
selecting a wide-angle lens whose angle of view as 
calculated in accordance with eq. 7 coincides with the 
blind spot* 
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As described above, according to the present 
invention , there is provided a highly-reliable 
omnidirectional vision sensor which does not require a 
special mechanical means and has substantially less blind 
spots than those associated with conventional 
omnidirectional vision sensors, such that information 
concerning a horizontal span covering 360 degrees around 
the sensor can be obtained in real-time. The 
omnidirectional vision sensor can be very effectively 
used in a vision system for a monitoring camera system 
or a mobile robot , etc . Since only one camera ( which tends 
to be expensive) is required, a vision sensor for making 
an omnidirectional observation can be realized at a 
relatively low cost. 

Various other modifications will be apparent to 
and can be readily made by those skilled in the art without 
departing from the scope and spirit of this invention. 
Accordingly, it is not intended that the scope of the 
claims appended hereto be limited to the description as 
set forth herein, but rather that the claims be broadly 
construed. 



